21 research outputs found
IronDepth: Iterative Refinement of Single-View Depth using Surface Normal and its Uncertainty
Single image surface normal estimation and depth estimation are closely
related problems as the former can be calculated from the latter. However, the
surface normals computed from the output of depth estimation methods are
significantly less accurate than the surface normals directly estimated by
networks. To reduce such discrepancy, we introduce a novel framework that uses
surface normal and its uncertainty to recurrently refine the predicted
depth-map. The depth of each pixel can be propagated to a query pixel, using
the predicted surface normal as guidance. We thus formulate depth refinement as
a classification of choosing the neighboring pixel to propagate from. Then, by
propagating to sub-pixel points, we upsample the refined, low-resolution
output. The proposed method shows state-of-the-art performance on NYUv2 and
iBims-1 - both in terms of depth and normal. Our refinement module can also be
attached to the existing depth estimation methods to improve their accuracy. We
also show that our framework, only trained for depth estimation, can also be
used for depth completion. The code is available at
https://github.com/baegwangbin/IronDepth.Comment: BMVC 202
A Neural Height-Map Approach for the Binocular Photometric Stereo Problem
In this work we propose a novel, highly practical, binocular photometric
stereo (PS) framework, which has same acquisition speed as single view PS,
however significantly improves the quality of the estimated geometry.
As in recent neural multi-view shape estimation frameworks such as NeRF,
SIREN and inverse graphics approaches to multi-view photometric stereo (e.g.
PS-NeRF) we formulate shape estimation task as learning of a differentiable
surface and texture representation by minimising surface normal discrepancy for
normals estimated from multiple varying light images for two views as well as
discrepancy between rendered surface intensity and observed images. Our method
differs from typical multi-view shape estimation approaches in two key ways.
First, our surface is represented not as a volume but as a neural heightmap
where heights of points on a surface are computed by a deep neural network.
Second, instead of predicting an average intensity as PS-NeRF or introducing
lambertian material assumptions as Guo et al., we use a learnt BRDF and perform
near-field per point intensity rendering.
Our method achieves the state-of-the-art performance on the DiLiGenT-MV
dataset adapted to binocular stereo setup as well as a new binocular
photometric stereo dataset - LUCES-ST.Comment: WACV 202
A CNN Based Approach for the Point-Light Photometric Stereo Problem
Reconstructing the 3D shape of an object using several images under different
light sources is a very challenging task, especially when realistic assumptions
such as light propagation and attenuation, perspective viewing geometry and
specular light reflection are considered. Many of works tackling Photometric
Stereo (PS) problems often relax most of the aforementioned assumptions.
Especially they ignore specular reflection and global illumination effects. In
this work, we propose a CNN-based approach capable of handling these realistic
assumptions by leveraging recent improvements of deep neural networks for
far-field Photometric Stereo and adapt them to the point light setup. We
achieve this by employing an iterative procedure of point-light PS for shape
estimation which has two main steps. Firstly we train a per-pixel CNN to
predict surface normals from reflectance samples. Secondly, we compute the
depth by integrating the normal field in order to iteratively estimate light
directions and attenuation which is used to compensate the input images to
compute reflectance samples for the next iteration.
Our approach sigificantly outperforms the state-of-the-art on the DiLiGenT
real world dataset. Furthermore, in order to measure the performance of our
approach for near-field point-light source PS data, we introduce LUCES the
first real-world 'dataset for near-fieLd point light soUrCe photomEtric Stereo'
of 14 objects of different materials were the effects of point light sources
and perspective viewing are a lot more significant. Our approach also
outperforms the competition on this dataset as well. Data and test code are
available at the project page.Comment: arXiv admin note: text overlap with arXiv:2009.0579